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This article details a case of using ePortfolios in the evaluation process and assessment of the 
Department of Communication at the University of Hawai‘i at Manoa. The program is guided by 
seven SLOs (student learning outcomes), which are demonstrable skills or abilities that students are 
expected to possess before receiving their degrees. The SLO framework was implemented in our 
department with the intent to promote effective learning through the application of a cohesive 
curriculum that was designed by faculty members. In 2013, we evaluated our program to assess its 
successes and shortfalls through ePortfolios as an assessment tool. The assessment findings noted 
gaps in our curriculum, along with a need to improve specific processes, such as better alignment of 
learning outcomes with the assessment rubric. Overall, we found that the ePortfolios and the 
assessment process in our senior capstone courses ensure the value of the curriculum over time and 
serve as agents for cultural change within the department. 


Culminating experiences, such as capstone courses 
or senior projects and seminars, represent extraordinary 
learning opportunities for college students. According 
to Kinzie (2013), capstone courses are designed as the 
final “integration of educational experiences and foster 
transition to work or further education beyond the 
bachelor’s degree experience” (p. 27). In the 2014 
National Survey of Student Engagement (NSSE, 2014) 
of 622 U.S. institutions, senior culminating experiences 
ranked third in student participation for high-impact 
educational practices, behind only service learning and 
field experiences. In addition, senior culminating 
experiences such as the capstone course provide more 
opportunities for diverse students (e.g., first generation 
students) to participate in high-impact education 
practices, as compared to study abroad and research 
with a faculty member (Kinzie, 2013). 

Although capstones are not necessarily a new 
phenomenon in higher education, there has been a 
rise in their importance as culminating experiences 
that accomplish both student development and 
program assessment (Berheide, 2007; Kinzie, 2013; 
Rowles, Koch, Hundley, & Hamilton, 2004). For 
student development, capstone courses provide 
opportunities to reflect on their own learning 
throughout their college experiences. This is done 
primarily through experiences that increase 
opportunities to “connect, deepen, and generalize 
learning beyond the immediate setting where it 
occurs” (Kinzie, 2013, p. 30). For program 
assessment, senior capstones provide key data to 
faculty regarding the quality of programs and 
instruction (Black & Hundley, 2004, p. 3). Student 
artifacts produced in senior capstone courses offer a 


direct, authentic, and efficient method for assessing 
how successful a curriculum is in addressing 
learning objectives (Berheide, 2007). One such 
artifact is the ePortfolio. 

In 2010, the School of Communications revised the 
undergraduate curriculum to reflect important developments 
that were occurring in the field of communication. Our 
decision to revise the curriculum was threefold. First, we 
acknowledged the move toward greater emphasis on 
mediated communication technologies and strategies 
(Lievrouw, 2009) and on how aspects of communication are 
altered in a digitally networked era (Papacharissi, 2011; 
Pfister & Soliz, 2011). A second reason was to elevate the 
importance of practical engagement experiences for our 
students (Jenkins, Purushotma, Weigel, Clinton, & Robison, 
2009). We sought to provide more educational experiences 
that connect to real-life work and service opportunities. The 
final reason was to stay within alignment of the University 
of Hawai‘i at Manoa’s strategic plan for improving student 
learning outcomes. The purpose of this paper is to detail our 
experiences with using ePortfolios in the implementation 
and assessment of a capstone course based on the revision 
of the School of Communications curriculum at the 
University of Hawai’i at Manoa. In detailing our case study, 
the information provided in detailing our case study will be 
of benefit to universities, colleges, and departments 
conducting program assessments with ePortfolios. 

ePortfolios in Higher Education 

Electronic portfolios have been employed by 
programs in higher education in a wide variety of 
instructional and assessment roles (Barrett, 2004, 2010; 
Sherman, 2006). Lorenzo and Ittelson (2005) defined 
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an electronic portfolio or ePortfolio as a “digitized 
collection of artifacts, including demonstrations, 
resources, and accomplishments that represent an 
individual, group, community, or organization’’ (p. 2). 
Jafari (2004) described ePortfolios as a content- 
management system that “facilitates the process of 
collecting, reflecting on, sharing, and presenting 
learning outcomes and other professional 
accomplishments via a digital medium” (p. 38). 
ePortfolios help to address program evaluation and 
accreditation concerns by providing an asset 
management system that facilitates a “framework for 
the uploading, organization, and accessing of artifacts” 
(Light, Chen, & Ittelson, 2012, p. 97). Clark and Eynon 
(2009) noted a growing interest in integrative learning 
focused on student experience as among the major 
drivers for ePortfolio use in higher education. 

Unlike an academic transcript, ePortfolios provide 
students with the opportunity to acknowledge learning 
that occurs outside the classroom through co-curricular 
or extra-curricular experiences. These formal or 
informal experiences may be on or off campus and can 
incorporate study abroad, community service, co-ops 
and internships. According to Light et al. (2012), it is 
important to capture these significant and meaningful 
experiences, since students may perceive value only in 
formal academic work. As a result, ePortfolios serve as 
a “context for integration of all learning as it occurs 
both inside and outside the classroom, but it can also 
make visible the internships, jobs, study abroad, and 
work in the community that are often opaque to faculty 
instructors” (p. 86). For example, Richards-Schuster, 
Ruffolo, Nicoll, Distelrath, and Galura (2014) found 
that students who participated in a capstone class for a 
community action and social change program 
demonstrated experiences of transformative and 
integrative civic engagement learning outcomes in their 
ePortfolio program assessment. Similarly, Kerrigan and 
Carpenter (2013) completed an ePortfolio assessment of 
capstone community partnership courses at Portland 
State University. Their findings revealed that students 
had a deepened sense of social responsibility and 
greater efficacy to serve as an advocate for underserved 
populations. 

A second driver in the adoption of ePortfolios is 
the rise of digital technologies used in higher education 
and the increased acknowledgment of user-generated 
content on the web (Clark & Eynon, 2009). The ease 
with which social media platforms allow students’ 
content to be created and shared leads to the formation 
of digital identities and a web presence. Ramirez (2011) 
described this as a hyper-inclusive ePerformance, in 
which students can potentially share and link 
ePortfolios to a limitless audience on the Web. A 
student’s digital content often becomes the first 
impression for future colleagues, employers, and dating 


partners. ePortfolios, unlike Facebook or Linkedln, 
allow students to decide, intentionally and thoughtfully, 
who should access their content (Jenson, 2011). 
Specifically, for communication majors, the ePortfolio 
asks students to “reflect on their construction of 
effective messages” (Whitfield, 2011, p. 241). 
According to Hoger (1998), this includes thinking 
critically about themselves and others as 
communicators and going “beyond the literal content of 
a message, perhaps to detect and act on subtext, to read 
between the lines, to consider larger contexts, to 
interpret innuendo, to detect strategic maneuvering, or 
to consider side effects” (p. 64). As opposed to 
Facebook, which provides a more limited view of an 
individual, ePortfolios represent evidence for the 
intersection of experiences, accomplishments, and 
reflections (Reynolds & Patton, 2014). In other words, 
the process and product of a learning ePortfolio is the 
“development of an intellectual identity, not a social 
identity” (Light et al., 2012, p. 74). However, Light et 
al. (2012) also noted that future technological 
innovations in the ePortfolio will most likely 
incorporate social media features such as commenting, 
links to Facebook or Twitter, improved multimedia 
capabilities, and the ability to tag artifacts or posts. 

Reynolds and Patton (2014) argued that students 
need to “manage their digital presence by creating a 
digital identity that reflects their values, skills, and 
accomplishments” (p. 102). In this vein, some 

researchers suggest that the ePortfolio process does lead 
to gains in developing a professional, digital identity 
(e.g., Peet et al., 2011). On the other hand, Snider and 
McCarthy (2012) revealed that the rigidity of an 
English ePortfolio system limited the flexibility for 
international students to craft truly personal and 
professional digital identities. 

A third driver is the increasing pressures for 
accountability and program assessment. As funding for 
public research universities continues to decline, 
programs must often demonstrate their value and 
effectiveness, and in some cases there are external 
accreditation issues. As higher education becomes 
increasingly focused on evidence of student learning, 
portfolios are seen as a valuable tool to “inform 
accreditation and accountability efforts” (Chen & Light, 
2010, p. 1). Chen and Light emphasized that this is 
especially important as a means to ensure curricular 
coherence in contexts where students have diverse 
learning experiences occurring both in and out of the 
classroom: 

As an assessment tool, the student portfolio is 
unique insofar as it captures evidence of student 
learning over time—in multiple formats and 
contexts—documents practice, and includes a 
student’s own reflection on his or her learning. 
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Portfolios also encourage students to represent and 

integrate their formal and informal learning 

experiences (Chen & Light, 2010, p. 1). 

They also noted that learning occurs in the process 
of portfolio creation. At the very least, assessment 
results can be shared with students, parents, and 
prospective employers to demonstrate the strengths of a 
program. 

A final reason is that ePortfolios provide a helpful 
framework for students to document and take 
ownership of their learning experiences. Students can 
easily share and connect their learning experiences with 
others through a digital networked environment. The 
ability to articulate and reflect on their achievements 
and demonstrate how these achievements relate to each 
other becomes a very useful skill for a job or graduate 
school interview (Light et al., 2012). Therefore, the 
capstone course and ePortfolio represent a new culture 
of learning in which students ask and answer their own 
questions, thereby managing their learning. 

As noted earlier, we provide our experience with 
implementing and assessing a capstone course and 
ePortfolio for the Communication BA at the University 
of Hawaii at Manoa. We outline the process of creating 
a program assessment process for a new curriculum 
designed by the Communication Department at the 
University of Hawaii at Manoa. We begin with a 
description of the learning environment at the 
University and the School of Communications and 
discuss specific student learning outcomes (SLOs) 
developed to guide program assessment. We then 
describe the new curriculum and the role of capstone 
ePortfolios for assessment strategies. 

Learning Environment at the University of Hawaii 
and School of Communications 

The University of Hawaii at Manoa is a land, sea, 
and space grant university. It is a research institution 
with a mission that focuses on service to the state of 
Hawaii and to both national and international 
communities. The Manoa campus offers 292 degree and 
certificate programs, bachelor’s through doctorate, and 
has a current student population of just over 20,000 
(University of Hawaii at Manoa, 2015). As part of the 
Manoa Strategic Plan for 2011-2015, a primary goal is 
a transformative learning environment, “build[ing] on 
the vision of education defined as the Manoa 
Experience, which provides students challenging and 
distinctive academic programs, innovative teaching and 
service, and world-class research and scholarship 
reflective of global perspectives and a culturally diverse 
island state” (University of Hawaii at Manoa, 2011, p. 
9). Focus on a revitalized undergraduate curriculum is 
further being pursued by the College of Social 


Sciences, which is actively working to strengthen the 
liberal arts curriculum (the School of Communications 
is part of the College of Social Sciences). In alignment 
with the goals of the Association of American Colleges 
and Universities’ (AAC&U, 2015a) national effort, 
Liberal Education and America’s Promise, the College 
has initiated its own strategy called Commitment to 
Liberal Education, focusing on high-impact educational 
practices such as collaborative projects, service 
learning, capstone courses, and learning communities. 

The Department of Communication offers one of 
two undergraduate programs in the School of 
Communications, as well as an MA program. It is also 
one of four departments, along with Information and 
Computer Science, Library and Information Science, 
and Management Information Systems, that sponsor the 
Interdisciplinary PhD Program in Communication and 
Information Sciences. There are approximately 235 
active Communication Department undergraduates, 
most declaring in their sophomore or junior years. 

Revised Communication Curriculum 

In the fall of 2008, the School of Communications 
hosted a two-day faculty workshop to create a revised 
curriculum for the Communication BA and MA 
programs that would more strongly prepare our 
graduates for productive careers and engaged 
citizenship in a complex, global, technology-mediated 
world. In particular, we sought to strengthen critical 
thinking skills for addressing complex, real-world 
situations, foster strong communication skills directed 
to a variety of audiences using different media, build 
awareness of both global and local issues, and develop 
the ability to work collaboratively in teams (often 
geographically distributed) and enable student 
engagement with both face-to-face and virtual 
communities of practice. These goals formed the basis 
of the student learning outcomes (SLOs), skills that we 
expect all of our graduates to master prior to entering 
the workforce or commencing graduate education. 
Table 1 lists the present outcomes that guide our 
instructional and assessment activities. 

Our revised curriculum was based on our 
understanding of the key traditions and evolution of the 
Communication field, our own faculty’s strengths, 
student interests, and feedback from alumni. We 
identified three separate, but interrelated, learning 
tracks and identified a set of common core courses for 
all majors and prerequisites for each track. Each track 
also has a focused capstone, a learning community that 
focuses on specific problems relevant to that content 
area. The AAC&U (2007) recognized capstones as one 
of ten high-impact educational practices that encourage 
deep learning. Because capstones require graduating 
students to create a personal project integrating what 
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Table 1 

Communication BA Program Student Learning Outcomes (SLO) 

SLO _ Criterion _ 

1 Design communication and media projects to make meaningful contributions to diverse social, 
professional, or academic communities, communicating effectively orally, in writing, and through digital 
media. 

2 Reflect critically on communication products such as media productions, research and policy reports, and 
everyday texts. 

3 Demonstrate preparedness for academic and professional careers in communication. 

4 Demonstrate global awareness, including an awareness of cultures in the Hawaii-Pacific region and issues 
related to cross-cultural communication. 

5 Engage in collaborative problem solving, both face-to-face and in online environments. 

6 Analyze the ethical dimensions of communication. 

7 _ Critically evaluate the use of technology in communication. _ 


they have learned, they foster reflection, holistic 
understanding, and transition to work or post-bachelor’s 
education (Kinzie, 2013). The revised curriculum was 
submitted to the university in 2009, and the curriculum 
changes were initiated in the fall of 2010. 

The first track, Communication in Communities, 
focuses on communication in social, organizational, and 
professional communities ranging from small groups 
working together face-to-face or online to large 
organizations communicating with international 
publics. Intercultural communication, international 
communication, organizational communication, and 
public relations are traditional academic areas of 
scholarship that inform this track. The Capstone in 
Communication in Communities involves project 
development within either local or global communities. 
Project options include a public relations campaign, an 
organizational communication audit, or a program for 
preparing, training, and supporting people in dealing 
with cultural diversity. Students explore specific project 
ideas in consultation with their professor and client 
organizations. 

A second track, Media Arts, includes two 
production sequences, Digital Cinema and Multimedia. 
Digital Cinema combines the learning of single-camera 
production skills in narrative with documentary-style 
filmmaking. Pre-production, production, and 
postproduction filmmaking skills are constructed with 
essential aesthetic values that go beyond technical 
application to theory, criticism, and cinema history. 
Multimedia combines visual communication theory and 
design aesthetics with digital media production 
knowledge such as digital still photography, time-based 
media, and web content design to convey information. 
Software programs used include Photoshop, Premiere 
Pro and other moving image, audio sound design, and 
interactive content management applications. Students 
in the Media Arts track may choose one of two 
capstone courses, Digital Cinema or Multimedia. 


The third track, ICTs & Policy, focuses on 
information and communication technologies (ICTs) 
and how they are shaped by, as well as influence, 
society. Students leam how ICTs such as the Internet, 
social media, mobile phones, online gaming and virtual 
worlds, digital video, peer-to-peer networks, and other 
emerging network technologies are used around the 
world. The Capstone Project in ICTs and Policy focuses 
on specific ICT and policy problems related to Hawaii 
and the Asia-Pacific region. There are several project 
options, including: a policy analysis presenting 
alternatives to address an issue related to ICTs; a 
traditional research project related to some aspect of 
ICTs; a project related to ICTs employing fiitures 
research methodologies; and an applied technology 
project with documentation. 

The capstone projects challenge students to 
demonstrate mastery of the communication curriculum 
by creating an original research project related to an 
area of interest within the track. A series of required 
courses ask students to become familiar with the 
research designs and methods used by communication 
scholars in their area of specialty; to understand the 
conceptual foundations, principles, practices, and 
traditions on which communication research is 
grounded; and to develop the ability to evaluate 
critically communication research that is presented in 
journals, scholarly texts, and visual media projects. The 
capstone projects encourage students to synthesize and 
reflect critically on their learning experiences, both in 
and out of the classroom. 

The capstones also involve the creation of an 
ePortfolio demonstrating mastery of our SLOs. While 
the specific projects vary, each capstone includes 
written reflection and electronic artifacts created in pre¬ 
requisite courses or through capstone assignments. 
Assignments from the four courses required of all 
majors are included in the portfolio, as well as track- 
specific projects. Faculty were in agreement regarding 
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the implementation of the senior capstone project as 
well as ePortfolios. 

Our curriculum is designed to promote authentic 
learning by challenging students to address “real-world, 
complex problems and their solutions, using role- 
playing exercises, problem-based activities, case 
studies, and participation in virtual communities of 
practice” (Lombardi, 2007, p. 2). Authentic learning 
focuses on the cultivation of portable skills by engaging 
students in real-world tasks and problems, focusing on 
ill-defined problems that require sustained 
investigation, and fostering metacognitive reflection. 
We approach problems from multiple perspectives and 
encourage interdisciplinary inquiry. Learning 
assessment is “woven seamlessly into the major task in 
a manner that reflects real-world evaluation processes” 
(Lombardi, 2007, p. 3). 

Integration of the ePortfolios was done 
incrementally beginning in the fall of 2010. First, we 
began to discuss the senior portfolio/capstone 
requirement in our required major courses. We 
requested buy-in from all tenured and non-tenured 
faculty. However, some faculty felt threatened by or 
anxious about the increase in transparency through the 
formal documentation of learning (Danley-Scott & 
Scott, 2014; Light et al., 2012). Majors became familiar 
with the rationale for assessment and learning 
portfolios, and they were encouraged to save class 
projects for later use. Our first capstones were offered 
in the spring of 2012. We offered five sections 
(including two of Communication and Communities 
and one each of the other capstones), and each senior 
created an ePortfolio as a course requirement. These 
portfolios contained an original project specific to a 
student’s track and personal interest, as well as at least 
four other artifacts from Communication courses. In 
addition, students were also required to provide a 
personal statement that incorporated elements of a 
reflective essay (Mummalaneni, 2014). By sifting 
through their various assignments and integrating 
knowledge from different Communication courses, 
students reflected on their own learning processes, 
encouraging metacognitive and critical thinking 
(Barrett, 2007; Zubizarreta, 2004). 

Several ePortfolio technologies were considered 
for the senior capstone courses. As our emphasis was 
on the students’ selection and creation of artifacts rather 
than on their learning a new technology, we decided to 
implement the portfolio functionality within Adobe 
Acrobat Pro, software that students were familiar with 
and that was readily available in our labs at no 
additional cost to learners. 

As noted by Clark and Eynon (2009), there is an 
increased focus on interdisciplinary learning in 
higher education. This includes a number of links 
between classroom activities, professional 


obligations, and students’ experiences outside of 
school. Our three tracks are not silos, so students 
are encouraged to select artifacts that reflect their 
interdisciplinary strengths as well as applied 
learning from internships or service-learning 
projects. In some cases, these experiences are via 
civic media, which offer the potential to bring 
together diverse communities at local, national, or 
global levels and encourage civic engagement 
among students (Rheingold, 2008; Jenkins et al., 

2009) . According to the National Task Force on 
Civic Learning and Democratic Engagement (2012), 
there is a vital need to “[ejxpand the number of 
robust, generative civic partnerships and alliances, 
locally, nationally, and globally to address common 
problems, empower people to act, strengthen 
communities and nations, and generate new 
frontiers of knowledge” (p. vi). 

In addition to providing evidence for program 
assessment, portfolio-building is a way to communicate 
our SLOs with students. Used as a point of reflection, 
our SLOs help learners to set personal and professional 
goals in relation to the curriculum. We share our 
assessment rubric with students in order to provide 
clear guidelines for scaffolding learning and enhancing 
students’ ability to do independent work (Bereiter & 
Scardamalia, 1987; Vygotsky, 1978). At the same time, 
we view learning as a social process and emphasize the 
role of students as active participants in social 
communities who are constructing identities by 
engaging in communities of practice (Wenger, 2000, 
2005). In the context of higher education, learning 
communities both link learning in communities around 
specific domains of interest and connect these 
experiences to broader communities outside of the 
classroom. 

Implementation of the assessment process. Our 

SLOs describe what we expect our Communication 
majors to be capable of before moving into the 
workforce or graduate education. However, for 
assessment purposes, we first needed to create 
measurable sets of performance criteria from our SLOs 
that are linked to portfolio components (Williams, 

2010) . These were, in turn, linked to an evaluation 
rubric. Our rubric was modified from several designed 
by the AAC&U (2015b). 

Our first capstones were offered in spring 2012 and 
represented the first full cycle of our new curriculum. 
We have assessed one track’s ePortfolios each year. We 
agreed to pilot our process by sampling one track per 
year for the first three years and then sampling from all 
tracks once the capstone portfolio process was fully 
integrated. This was to provide additional time for 
instructors to integrate the ePortfolio into their capstone 
courses. A panel of faculty from other tracks, 
prospective employers, and alumni took part in the 
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evaluation, using rubrics employing measurable items 
associated with our seven SLOs. We see this 
assessment panel as an opportunity to strengthen ties 
with the community and, in particular, civic groups, 
potential employers, and alumni. This also provides 
feedback from various stakeholders about curriculum 
design. In this way, we hope to create “a feedback loop 
that serves to update the academy on the skills required 
by students as they enter society” (Acosta & Liu, 2006, 
P-18). 

In 2013, all 40 ePortfolios from the 
Communication in Communities track were reviewed in 
order to assess the degree to which we have met our 
program SLOs. As discussed previously, ePortfolios 
serve as digital content management systems. As such, 
case study approaches have been described in various 
studies in information systems (Lee, Liebenau, & 
DeGross, 1997). The data for our study comes from 
various sources: analysis of student portfolio artifacts 
via data collected though a Qualtrics online survey 
completed by the assessment panel, panel feedback 
about alignment with artifacts with SLOs and about the 
rubric, and notes from the full Communication faculty 
discussion about the panel’s findings. Data from all of 
these sources comprised our “database” to formalize the 
organization and analysis (Yin, 2011). 

In addition to four faculty members from the 
Communication Department, we invited two members 
of our Advisory Board, alumni with high-profile jobs in 
our field. In fall 2013, the six-member panel met for an 
orientation session, and each panelist was given a 
packet explaining the process. It was important to 
remind panelists that we were assessing the B.A. 
program, not specific students, instructors, or classes. 
Packets included copies of the rubric and a CD-ROM 
with a subset of the portfolios. Because there were 40 
portfolios, we gave one third (either 13 or 14) to each 
panelist, and each portfolio was assessed by two people. 

As noted by Bcrheide (2007), a priority for 
capstone assessment is to minimize additional work for 
faculty. Since this was our first round of assessment 
after the curriculum change and to ease any faculty 
apprehension, our focus was on streamlining the 
assessment procedures. To this end, the assessment 
coordinator explained the rubric and provided helpful 
portfolio examples to clarify correct application and 
alignment of student artifacts to SLOs. In addition, 
since we were employing the online survey software 
Qualtrics to input scores, the coordinator demonstrated 
how to access the website and input assessment data. 

Senior capstone projects. Senior capstone 
projects represented key artifacts for the ePortfolio, 
highlighting practical engagement experiences that 
incorporate service learning to assist Oahu nonprofits. 
Students worked in teams to create a business plan for 
those nonprofits that agreed to participate in the 


capstone experience and were tasked with the 
following: (1) do research on the history of the 
organization; (2) examine their social media presence 
from a public relations perspective; (3) evaluate current 
communication plans and procedures; and (4) provide 
suggestions to the client for improvement. Working 
with real-world clients helped students to connect their 
learning and experiences from the Communication 
program to specific goals and needs by local 
organizations. Notable clients included the Waikiki 
Aquarium, Surfrider Foundation Oahu chapter, and 
Native Hawaiian Student Services at the University of 
Hawaii at Manoa. 

After the orientation, panelists were given two 
weeks to review the portfolios assigned to them and 
input the scores into the Qualtrics online scoresheet. 
Once the scores were collected, the assessment 
coordinator constructed a summary table showing each 
SLO and the distribution (percentage) of students that 
were given each score. This was brought to the second 
meeting of the panel for review. During the second 
meeting, we gathered additional, qualitative assessment 
of students’ portfolios and also highlighted areas for 
improvement, both within the curriculum and the 
assessment process. A formal assessment report was 
created from this process. This was shared in advance 
with our entire Communication faculty, and a 
department-wide meeting was called to discuss the 
curriculum. The Assessment Coordinator created a 
summary report and shared it with Communication 
faculty at an assessment meeting in November. We set 
an initial benchmark of 80% of students scoring as 
either proficient or exemplary for each SLO. Results for 
each SLO are presented below (Table 2). 

Additionally, we investigated inter-rater 
consistency on the ratings given to 40 students on seven 
separate SLOs. Two indicators were used to represent 
inter-rater consistency: (1) inter-rater agreement rate, 
which is the percentage of identical ratings given by 
two raters for each SLO; and (2) inter-rater agreement 
within one-point difference, which is the percentage of 
the ratings given by two raters on each SLO that differ 
by one point or are identical. Each rater was supposed 
to give one rating for each of the seven SLOs, using 
student work aligned with that SLO. Each student’s 
work under one SLO was evaluated by two raters. This 
means that the total possible number of paired ratings 
was 40 per SLO. However, some works had missing 
ratings from one or two raters. Table 3 shows the 
number of paired ratings that we used to calculate inter¬ 
rater agreement for each SLO. It also shows the inter¬ 
rater agreement rate, and inter-rater agreement rate 
within one-point difference. 

In general, raters gave very similar ratings. Their 
ratings were the same or only differed by one point over 
85% of the times on student works related to all SLOs 
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Table 2 


Percent Distribution of Assessment Scores by SLO with Percentage Achieving Benchmarks (N = 40) 


SLO 

Summary 

Unacceptable 

Marginal 

Proficient 

Exemplary 

Benchmark 

1 

Effective project design/ 
communication* 

3 

8 

55 

35 

90 

2 

Critical thinking 

0 

5 

77 

18 

95 

3 

Career readiness 

0 

15 

65 

20 

85 

4 

Global/intercultural 

awareness 

5 

18 

51 

26 

77 

5 

Collaborative problem 
solving 

3 

0 

69 

28 

97 

6 

Ethical deliberation 

5 

37 

42 

16 

58 

7 

Critical evaluation of ICTs 

3 

18 

67 

13 

80 


Note : All values are percentages. SLO Benchmark percentages in boldface have approximated or exceeded the 80% threshold. 


Table 3 


Inter-Rater Consistency by Number of Paired Ratings and Percentage of Inter-Rater Agreement (N=40) 





Inter-rater agreement 

SLO 

Number of paired ratings 

% of agreement 

% of agreement 
(+1 or -1 difference) 

1 

40 

50% 

95% 

2 

37 

52% 

92% 

3 

40 

53% 

90% 

4 

35 

40% 

91% 

5 

31 

61% 

87% 

6 

34 

29% 

68% 

7 

37 

32% 

89% 


except for SLO 6 (68%). The strict inter-rater 
agreement rates were much lower, ranging between 
29% on SLO 6 to 61% on SLO 5. These results indicate 
that raters were able to give similar ratings, but more 
rater training will enhance rating consistency. 

To increase the quality of data, a third rater 
evaluated all the student works with ratings that were 
more than one point apart. The outlier’s ratings were 
discarded from the analysis. In the end, there was 100% 
inter-rater agreement within one-point difference 

Overall, our graduates are meeting or exceeding 
our expectations. The two areas that we highlighted for 
improvement related to ethical deliberation (58%) and 
intercultural and global awareness (77%). We noted 
that some alignment issues were found between the 
artifacts presented and the rubric (i.e., students may 
have chosen works that were not ideal matches), so 
these data should be considered in light of this 
limitation. We discuss these results below. 

Discussion 

The revision of the Communications curriculum 
was motivated by recognition that our students needed 
additional preparation for a more complex, global, 


computer-mediated world. Specifically, our goals 
reflect careful consideration of desired student learning 
outcomes where the ePortfolio can contribute to 
significant and meaningful teaching, learning, and 
assessment. In addition, we assessed the 
Communication program by sampling from ePortfolio 
submissions in the Communication and Communities 
track. Communication and Communities focuses on 
social, organizational, and professional communities, 
from large to small and in either face-to-face or 
mediated settings. Students sampled during this 
timeframe met or surpassed the benchmark (80%) in all 
but two areas. This provides evidence to support our 
expectation that student ePortfolios demonstrate almost 
all the learning outcomes in our curriculum. In a recent 
literature review of ePortfolio research, Bryant and 
Chittum (2013) concluded that there is a greater need to 
present original data on student outcomes through 
ePortfolio use. We believe our study helps to contribute 
to this need by assessing the ePortfolio’s effect on 
communication students’ learning outcomes. 

As noted by Fitch, Peet, Reed, and Tolman (2008), 
our faculty “did not assume that all competencies are 
captured in student written assignments” (p. 47). In this 
regard, digital cinema and multimedia projects, student 
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presentations, and group work may also effectively 
represent student competencies but are more difficult to 
place as an ePortfolio artifact and to be appropriately 
acknowledged in the accompanying rubric. Multimedia 
assessment is a concern in ePortfolio research, since 
“the application of rubrics for assessing multimedia 
examples of student work collected via ePortfolios is 
currently being explored” (Light et al., 2012, p. 99). 
This is of particular importance in the communications 
field, where students learn and/or complete various 
multimedia campaign strategies, projects, and 
performances both in and outside of the classroom. As 
a result, we plan to integrate more assignments that 
focus on digital presence, purpose, and audience in the 
curriculum (Reynolds & Patton, 2014). 

Policy Changes Based on the ePortfolio Assessment 

The full faculty meeting held to discuss the 
findings of the assessment panel allowed for a great 
deal of informal discussion and novel idea generation. 
It also provided an opportunity for faculty to learn 
about each other’s courses and modify content to better 
foster student learning (e.g., identifying gaps in the 
curriculum or areas where strategic reinforcement of 
content across multiple courses would be beneficial). 
We used the end of the meeting to prioritize a few 
changes that are expected to have the greatest positive 
impact on student learning. These include the 
following: 

• Introduction of a hallmark assignment for 
each track (as part of the required course for 
that track). Because the current process 
allows a great deal of flexibility in artifact 
selection, a signature hallmark assignment 
will allow comparisons of student learning 
across semesters. 

• Integration of more assignments related to 
ethical considerations throughout the 
curriculum, along with an update of our 
curriculum map to demonstrate this 
competency. The curriculum map is a visual 
display of all courses that shows where in 
the curriculum the SLOs are introduced, 
reinforced, or mastered. This enables us to 
provide an appropriate sequence of learning 
experiences to address all SLOs 

• Creating strategies to assist 
students/instructors in the selection of 
artifacts (to ensure alignment). We found 
that many artifacts presented as evidence 
were not well matched to the corresponding 
SLO. Thus, we focused attention on 
clarifying selection procedures for both 
faculty and students. 


• Further rubric revision to enhance clarity and 
alignment. Our panel meeting and subsequent 
discussion also highlighted aspects of the 
rubric that panelists found problematic. For 
example, if there are multiple criteria at each 
level of the rubric, what happens if a student 
provides exemplary evidence for some, but not 
all? We addressed this by making some small 
changes to the rubric text and also clarifying 
the instructions given to the panelists. 

Communicating the Results 

We have several audiences for our results, and each 
required a different, strategic message. First, as noted 
above, our entire faculty received and discussed the 
results of the assessment panel. This led to enhanced 
coordination of classes and revision of content that 
improved the coherence of the curriculum. A second 
audience was our students. For prospective students, it 
is important to convey our results to show what they 
can anticipate learning in our program. We intend to 
feature future results on our department’s website, as 
part of an overall site redesign. This message is also 
important to share with current students, as it helps 
them to make sense of individual course objectives (that 
are linked explicitly to our SLOs). Instructors are 
encouraged to talk about the SLOs and our success in 
meeting them in courses throughout the program. A 
third audience is university administration, which 
gauges the health and success of our program based on 
our assessment data. We provide a summary assessment 
report each fall, and this is posted publicly on the 
Assessment Office’s website. We also presented our 
preliminary findings at a campus-wide poster session 
focused on assessment for curricular improvement. A 
final audience is our alumni and prospective employers, 
two groups that often overlap. Our strategy for reaching 
this group is similar to that for prospective students— 
we will have updated assessment results featured on our 
website after its redesign. 

Conclusion 

In this paper, we have detailed the case of our 
experiences with using ePortfolios in the 
implementation and assessment of the Communication 
BA in the School of Communications curriculum at the 
University of Hawaii at Manoa. We also presented the 
results of our pilot project integrating capstone 
ePortfolios for student learning and program 
assessment. As a field, communication is 
interdisciplinary, and we found that the ePortfolio can 
help make relational connections within our tracks, 
providing a more coherent learning experience that also 
integrates classroom experiences with real-life work 
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and service opportunities. We described the learning 
environment at the University and the School of 
Communications and presented the specific student 
learning outcomes (SLOs) that we developed to guide 
and assess learning outcomes. We then elaborated the 
process of implementing our assessment plan, and 
presented the results of a recent program evaluation. 
We found that student ePortfolios demonstrated that 
students met our benchmarks for five of seven learning 
outcomes in our curriculum. We also discussed how we 
used our results to strengthen our curriculum and how 
we conveyed our assessment results to different 
stakeholders. The assessment findings noted gaps in our 
curriculum and the need for improving specific 
processes, such as better alignment of learning 
outcomes with the assessment rubric. Overall, we found 
that the process ensures the value of the curriculum 
over time and serves as an agent for cultural change 
within the department. 
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